Method and apparatus for adapting to gaming venue states

ABSTRACT

A system that incorporates teachings of the present disclosure may include, for example, a method for detecting that a gaming system has presented a first of a plurality of gaming venue states of a game, presenting a first of a plurality of screen shots at a display responsive to detecting the first gaming venue state, detecting that the gaming system has transitioned to a presentation of a second of the plurality of venue states, and presenting a second of the plurality of screen shots at the display responsive to detecting the second gaming venue state. Additional embodiments are disclosed.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of and claims priority to U.S. patentapplication Ser. No. 14/040,221, filed Sep. 27, 2013, which is acontinuation and claims priority to U.S. patent application Ser. No.13/210,857 (Now U.S. Pat. No. 8,562,435), filed Aug. 16, 2011. Theforgoing are incorporated herein by reference in their entirety.

FIELD OF THE DISCLOSURE

The present disclosure relates generally to a method and apparatus foradapting to gaming venue states.

BACKGROUND

It is common today for gamers to utilize more than one gaming accessory.This is especially true of gamers who play Massively Multiplayer On-line(MMO) games in a team or individual configuration. Gamers can have attheir disposal accessories such as a keyboard, a general purpose gamingpad, a mouse, a gaming console controller, a headset with a built-inmicrophone to communicate with other players, a joystick, a computerconsole, or other common gaming accessories.

A gamer can frequently use a combination of these accessories in onegame (e.g., headset, a keyboard, and mouse). Efficient management andutilization of these accessories can frequently impact a gamer's abilityto compete.

Accessory management can have utility in other disciplines which may notrelate to gaming applications. Efficient use of accessories in theseother disciplines can be important to gamers.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 depicts an illustrative embodiment of a Graphical User Interface(GUI) generated by an Accessory Management Software (AMS) applicationaccording to the present disclosure;

FIG. 2 depicts an embodiment for programming an apparatus with the AMSapplication;

FIG. 3 depicts illustrative embodiments for utilizing the apparatus;

FIG. 4 depicts an embodiment without the use of the apparatus;

FIG. 5 depicts an illustrative embodiment of a block diagram of acommunication device;

FIGS. 6-7 depict illustrative embodiments of the GUI of FIG. 1;

FIGS. 8-10 depict illustrative methods describing the operation of theAMS application;

FIGS. 11-12 depict illustrative embodiments of a tablet and gamingconsole that operates according to the method of FIG. 10; and

FIG. 13 depicts an illustrative diagrammatic representation of a machinein the form of a computer system within which a set of instructions,when executed, may cause the machine to perform any one or more of themethodologies disclosed herein.

DETAILED DESCRIPTION

One embodiment of the present disclosure can entail an apparatus havinga memory coupled to a controller. The controller can be operable toidentify a plurality of screen shots, where each screen shot comprisinga plurality of selectable elements for presentation at a touch screendisplay, and where each selectable element when selected by way of thetouch screen display generates a stimulation. The controller can also beoperable to associate at least a portion of the stimulations generatedwhen selecting the plurality of selectable elements with one or moresubstitute accessory stimulations, associate the plurality of screenshots with a plurality of gaming venue states, and record the pluralityof screen shots and the associations with the plurality of gaming venuestates and the one or more substitute accessory stimulations. Thecontroller can be further operable to establish communications with avideo gaming system that is engaged in presenting a video game, detectthat the video gaming system has presented a first of the plurality ofgaming venue states, identify a first of the plurality of screen shotsassociated with the first gaming venue state, and present the firstscreen shot at the touch screen display. The controller can also beoperable to receive from the touch screen display a first stimulationresponsive to a selection of one of the plurality of selectable elementsof the first screen shot, detect a recorded association between thefirst stimulation and a first of one or more substitute accessorystimulations, and transmit the first of the one or more substituteaccessory stimulations to the video gaming system to control apresentation of the video game.

One embodiment of the present disclosure can entail a computer-readablestorage medium having computer instructions to detect that a videogaming system has presented a first of a plurality of gaming venuestates of a video game, identify a first of a plurality of screen shotsassociated with the first gaming venue state, and present the firstscreen shot at a touch screen display. The storage medium can also havecomputer instructions to receive from the touch screen display a firststimulation responsive to a selection of one of a plurality ofselectable elements of the first screen shot, detect an associationbetween the first stimulation and a first of one or more substitutestimulations, and transmit the first of the one or more substitutestimulations to the video gaming system to control a presentation of thevideo game.

One embodiment of the present disclosure can entail a method fordetecting that a gaming system has presented a first of a plurality ofgaming venue states of a game, presenting a first of a plurality ofscreen shots at a touch screen display responsive to detecting the firstgaming venue state, detecting that the gaming system has transitioned toa presentation of a second of the plurality of venue states, andpresenting a second of the plurality of screen shots at the touch screendisplay responsive to detecting the second gaming venue state. The firstscreen shot can have a first plurality of selectable elements forpresentation at the touch screen display, where each selectable elementof the first screen shot when selected by way of the touch screendisplay generates a first of a plurality of stimulations for controllingoperations of the game. The second screen shot can have a secondplurality of selectable elements for presentation at the touch screendisplay, where each selectable element of the second screen shot whenselected by way of the touch screen display generates a second of theplurality of stimulations for controlling operations of the game.

FIGS. 1, 5 and 6 depict illustrative embodiments of a Graphical UserInterface (GUI) generated by an Accessory Management Software (AMS)application according to the present disclosure. The AMS application canoperate in a computing device such as a desktop computer, a laptopcomputer, a server, a mainframe computer, a gaming console, a gamingaccessory, or combinations thereof. The AMS application can also operatein other computing devices with computing resources such as a cellularphone, a personal digital assistant, or a media player (such as aniPOD™). From these illustrations it is contemplated that the AMSapplication can operate in any device with suitable computing resources.

FIGS. 2-3 depict a portable apparatus 202 programmable by the AMSapplication. FIG. 2 depicts an illustration of how the portableapparatus 202 can be programmed. In this illustration, the portableapparatus can take the form of a Universal Serial Bus (USB) dongle(herein referred to as dongle 202). The dongle 202 can becommunicatively coupled to a computer 204 by a physical interface 206such as a USB port or wireless interface 208 such as Bluetooth orWireless Fidelity (WiFi). The computer 204 can be used to executeoperations of the AMS application. In this configuration, the AMSapplication can be directed to generate associations between accessorystimuli and input functions of accessories which can be programmed inthe dongle 202 as will be described below.

FIG. 3 illustrates a number of embodiments for utilizing the dongle 202with a gaming accessory 302 (in this illustration a touch screen displaytablet, herein referred to as tablet 302) and a computing device 306such as a gaming console or a computer capable of presenting gamingapplication (herein referred to as gaming console 306). In theillustration of FIG. 3, the USB portion of the dongle 202 can bephysically engaged with either the tablet 302 or the gaming console 306.The dongle 202 in either of these configurations can communicate withthe tablet 302 or the gaming console 306 by wireless means 304 (e.g.,WiFi or Bluetooth). Other embodiments of the dongle 202 arecontemplated. For instance, the functions in whole or in part of thedongle 202 can be an integral part of the tablet 302 and/or the gamingconsole 304. In FIG. 4, the tablet 302 can alternatively be tethered tothe gaming console 304 by cable (e.g., USB cable) to provide a means ofcommunication less susceptible to electromagnetic interference or othersources of wireless interferences.

FIG. 5 depicts an illustrative embodiment of a communication device 500.Communication device 500 can serve in whole or in part as anillustrative embodiment of the devices depicted in FIGS. 1-4. Thecommunication device 500 can comprise a wireline and/or wirelesstransceiver 502 (herein transceiver 502), a user interface (UI) 504, apower supply 514, a location receiver 516, a motion sensor 518, anorientation sensor 520, and a controller 506 for managing operationsthereof. The transceiver 502 can support short-range or long-rangewireless access technologies such as Bluetooth, WiFi, Digital EnhancedCordless Telecommunications (DECT), or cellular communicationtechnologies, just to mention a few. Cellular technologies can include,for example, CDMA-1X, UMTS/HSDPA, GSM/GPRS, TDMA/EDGE, EV/DO, WiMAX,software defined radio (SDR), Long Term Evolution (LTE), as well asother next generation wireless communication technologies as they arise.The transceiver 502 can also be adapted to support circuit-switchedwireline access technologies (such as PSTN), packet-switched wirelineaccess technologies (such as TCP/IP, VoIP, etc.), and combinationsthereof.

The UI 504 can include a depressible or touch-sensitive keypad 508 witha navigation mechanism such as a roller ball, a joystick, a mouse, or anavigation disk for manipulating operations of the communication device500. The keypad 508 can be an integral part of a housing assembly of thecommunication device 500 or an independent device operably coupledthereto by a tethered wireline interface (such as a USB cable) or awireless interface supporting for example Bluetooth. The keypad 508 canrepresent a numeric keypad commonly used by phones, and/or a QWERTYkeypad with alphanumeric keys. The UI 504 can further include a display510 such as monochrome or color LCD (Liquid Crystal Display), OLED(Organic Light Emitting Diode) or other suitable display technology forconveying images to an end user of the communication device 500.

In an embodiment where the display 510 is touch-sensitive, a portion orall of the keypad 508 can be presented by way of the display 510 withnavigation features. As a touch screen display, the communication device500 can be adapted to present a user interface with graphical userinterface (GUI) elements that can be selected by a user with a touch ofa finger. The touch screen display 510 can be equipped with capacitive,resistive or other forms of sensing technology to detect how muchsurface area of a user's finger has been placed on a portion of thetouch screen display. This sensing information can be used control themanipulation of the GUI elements.

The UI 504 can also include an audio system 512 that utilizes commonaudio technology for conveying low volume audio (such as audio heardonly in the proximity of a human ear) and high volume audio (such asspeakerphone for hands free operation). The audio system 512 can furtherinclude a microphone for receiving audible signals of an end user. Theaudio system 512 can also be used for voice recognition applications.The UI 504 can further include an image sensor 513 such as a chargedcoupled device (CCD) camera for capturing still or moving images.

The power supply 514 can utilize common power management technologiessuch as replaceable and rechargeable batteries, supply regulationtechnologies, and charging system technologies for supplying energy tothe components of the communication device 500 to facilitate long-rangeor short-range portable applications. Alternatively, the charging systemcan utilize external power sources such as DC power supplied over aphysical interface such as a USB port.

The location receiver 516 can utilize location technology such as aglobal positioning system (GPS) receiver capable of assisted GPS foridentifying a location of the communication device 500 based on signalsgenerated by a constellation of GPS satellites, thereby facilitatingcommon location services such as navigation. The motion sensor 518 canutilize motion sensing technology such as an accelerometer, a gyroscope,or other suitable motion sensing technology to detect motion of thecommunication device 500 in three-dimensional space. The orientationsensor 520 can utilize orientation sensing technology such as a compassto detect the orientation of the communication device 500 (North, South,West, East, combined orientations thereof in degrees, minutes, or othersuitable orientation metrics).

The communication device 500 can use the transceiver 502 to alsodetermine a proximity to a cellular, WiFi, Bluetooth, or other wirelessaccess points by common sensing techniques such as utilizing a receivedsignal strength indicator (RSSI) and/or a signal time of arrival (TOA)or time of flight (TOF). The controller 506 can utilize computingtechnologies such as a microprocessor, a digital signal processor (DSP),and/or a video processor with associated storage memory such as Flash,ROM, RAM, SRAM, DRAM or other storage technologies.

Other components not shown in FIG. 5 are contemplated by the presentdisclosure. For instance, the communication device 500 can include areset button (not shown). The reset button can be used to reset thecontroller 506 of the communication device 500. In yet anotherembodiment, the communication device 500 can also include a factorydefault setting button positioned below a small hole in a housingassembly of the communication device 500 to force the communicationdevice 500 to re-establish factory settings. In this embodiment, a usercan use a protruding object such as a pen or paper clip tip to reachinto the hole and depress the default setting button.

The communication device 500 as described herein can operate with moreor less components described in FIG. 5. These variant embodiments arecontemplated by the present disclosure.

FIGS. 8-10 depict illustrative methods 800-1000 describing the operationof the AMS application and the apparatus 202. Method 800 can begin withstep 802 in which the AMS application is invoked in a computing device.The computing device can be the gaming console 306 and the tablet 302 ofFIGS. 3 and 4, or any other suitable computing device that can interactor execute a gaming application such as a video game. The invocationstep can result from a user selection of the AMS application from a menuor iconic symbol presented on a desktop of the computing device by anoperating system (OS) managing operations thereof. In step 804, the AMSapplication can detect by way of drivers in the OS a plurality ofoperationally distinct accessories communicatively coupled to thecomputing device. The accessories can be coupled to the computing deviceby a tethered interface (e.g., USB cable), a wireless interface (e.g.,Bluetooth or Wireless Fidelity—WiFi), or combinations thereof.

In the present context, an accessory can represent any type of devicewhich can be communicatively coupled to the computing device (or anintegral part of the computing device) and which can control aspects ofthe OS and/or a software application operating in the computing device.An accessory can represent for example a keyboard, a touch screendisplay, a gaming pad, a mouse, a gaming console controller, a joystick,a microphone, or a headset with a microphone—just to mention a few. Thekeyboard and touch screen display represent accessories of a similarcategory since their operational parameters are alike.

A mouse, on the other hand, represents an accessory having disparateoperational parameters from the keyboard or touch screen display. Forinstance, the operational parameters of a keyboard generally consist ofalphanumeric keys, control keys (e.g., Shift, Alt, Ctrl), and functionkeys while the operational parameters of a mouse consist of navigationdata generated by a tracking device such as a laser sensor, buttons toinvoke GUI selections, and settings thereof (e.g., counts or dots perinch, acceleration, scroll speed, jitter control, line straighteningcontrol, and so on). Such distinctions can be used to identify disparatecategories of accessories.

In step 806, the AMS application presents a GUI 101 such as depicted inFIG. 1 with operationally distinct accessories such as the keyboard 108and touch screen display 115. The GUI 101 presents the accessories108-116 in a scrollable section 117. One or more accessories can beselected by a user with a common mouse pointer. In this illustration,the keyboard 108 and touch screen display 115 were selected with a mousepointer for customization. Upon selecting the keyboard 108 and the touchscreen display 115 in section 117, the AMS application presents thekeyboard 108 and touch screen display 115 in split windows 118, 120,respectively, to help the user during the customization process.

In step 808, the AMS application can be programmed to detect auser-selection of a particular software application such as a game. Thisstep can be the result of the user entering in a Quick Search field 160the name of a gaming application (e.g., World of Warcraft™). Uponidentifying a gaming application, the AMS application can retrieve instep 810 from a remote or local database gaming application actionswhich can be presented in a scrollable section 139 of the GUIrepresented as “Actions” 130. The actions can be tactical actions 132,communication actions 134, menu actions 136, and movement actions 138which can be used to invoke and manage features of the gamingapplication.

The actions presented descriptively in section 130 of the GUI canrepresent a sequence of accessory input functions which a user canstimulate by button depressions, navigation or speech. For example,depressing the left button on the mouse 110 can represent the tacticalaction “Reload”, while the simultaneous keyboard depressions “Ctrl A”can represent the tactical action “Melee Attack”. For ease of use, the“Actions” 130 section of the GUI is presented descriptively rather thanby a description of the input function(s) of a particular accessory.

Any one of the Actions 130 can be associated with one or more inputfunctions of the accessories by way of a simple drag and drop action.For instance, a user can select a “Melee Attack” by placing a mousepointer 133 over an iconic symbol associated with this action. Upondoing so, the symbol can be highlighted to indicate to the user that theicon is selectable. At this point, the user can select the icon byholding the left mouse button and drag the symbol to any of the inputfunctions (e.g., buttons) of the keyboard 108 or selectable GUI elements119 of the touch screen display 115 to make an association with an inputfunction of one of these accessories. Actions of one accessory can beassociated with another accessory that is of a different category. Forexample, actions of a mouse or keyboard can be associated withselectable GUI elements 119 of the touch screen display 115.

The selectable GUI elements 119 can also be customized by the AMSapplication. For example, GUI elements 119 can be created withrectangular, square, circular, oval or other geometric shapes. Each GUIelement 119 can be given a name, and/or a symbol or thumbnail can beenclosed in the shape of the GUI element 119. The association of actions130 with the GUI elements 119 can be performed by the drag and dropoperations described above.

The AMS application can also be operable to store a plurality of screenshots of the GUI elements 119 of the touch screen display 115. Eachscreen shot can represent a set of selectable GUI elements 119 which areappropriate for a specific gaming venue state of a game (such as a videogame). A video game can have a plurality of gaming venue states. Agaming venue state can represent a state in which the avatar of a gamerhas jumped into a tank. In this gaming venue state, the screen shot fortank controls would be appropriate for presentation at the touch screendisplay 115. Suppose that the gamer chooses to direct the avatar toleave the tank and proceed on foot to another location. This canrepresent yet another gaming venue state. In this state, a screen shotwith GUI elements for carry-on weapons, ammunition, and other gamingaspects would be useful to the player. Suppose next that the gamerdirects the avatar to enter a helicopter. This can represent anothergaming venue state. In this state, a screen shot with GUI elements forcontrolling operations of the helicopter would be useful to the gamer.

Each of these screen shots can be created by manual customization asdirected by the gamer when creating GUI elements using the AMSapplication as described above. Alternatively, or in combination, theGUI elements can be supplied by the author of the gaming application viaan Internet download request, information provided on a gaming CD, orscreen shots provided while the gaming application is in operation. Itshould be further noted that the GUI elements in some instances may notbe selectable, but rather informative. For example, the GUI elements canbe used to describe a gamer's health (e.g., health of the avatar) by acolor spectrum. A GUI element can be also used to show a portion of thegame as static images (e.g., map with indications of where the enemy maybe), or moving images (e.g., video portions of the game).

Each of the screen shots can be associated with gaming venue states andactions from the Action menu 130. For example, the user can drag theMelee Attack symbol to the “Attack” touch screen button of the touchscreen display 115 thereby causing an association between the selectionof the “Attack” button and the gaming action of a Melee Attack. When the“Attack” button of the touch screen display 115 is selected duringnormal operation, the AMS application can detect the selection as a“trigger” to generate the key sequence “Ctrl A” which is understood bythe gaming application as a request for a Melee Attack. The gamingapplication operating in the gaming console 306 receives the “Ctrl A”sequence as if it had been generated by a QWERTY keyboard.

With this in mind, attention is directed to step 812 where the AMSapplication can respond to a user selection of a profile. A profile canbe a device profile or master profile invoked by selecting GUI button156 or 158, each of which can identify the association of actions withinput functions of one or more accessories. If a profile selection isdetected in step 812, the AMS application can retrieve in step 814macro(s) and/or prior associations of actions with the accessories asdefined by the profile. The actions and/or macros defined in the profilecan also be presented in step 816 by the AMS application in the actionscolumn 130 of the GUI 101 to modify or create new associations.

In step 818, the AMS application can also respond to a user selection tocreate a macro. A macro in the present context can mean any actionablecommand which can be recorded by the AMS application. An actionablecommand can represent a sequence of input functions of an accessory,identification of a software application to be initiated by an operatingsystem (OS), or any other recordable stimulus to initiate, control ormanipulate software applications. For instance, a macro can represent auser entering the identity of a software application (e.g., instantmessaging tool) to be initiated by an OS.

A macro can also represent recordable speech delivered by a microphonesingly or in combination with a headset for detection by anothersoftware application through speech recognition or for delivery of therecorded speech to other parties. In yet another embodiment a macro canrepresent recordable navigation of an accessory such as a mouse orjoystick, recordable selections of buttons on a keyboard, a mouse, or amouse pad, and so on. Macros can also be combinations of the aboveillustrations. Macros can be created from the GUI 101 by selecting a“Record Macro” button 148. The macro can be given a name and category inuser-defined fields 140 and 142.

Upon selecting the Record Macro button 148, a macro can be generated byselection of input functions on an accessory (e.g., Ctrl A, speech, GUIelement 119, etc.) and/or by manual entry in field 144 (e.g., typing thename and location of a software application to be initiated by an OS,such as an instant messaging application). Once the macro is created, itcan be tested by selecting button 150 which can repeat the sequencespecified in field 144. The clone button 152 can be selected toreplicate the macro sequence if desired. Fields 152 can also presenttiming characteristics of the stimulation sequence in the macro with theability to modify and thereby customize the timing of one or morestimulations in the stimulation sequence. Once the macro has been fullydefined, selection of button 154 records the macro in step 820. Therecording step can be combined with a step for adding the macro to theassociable items Actions column 130, thereby providing the user themeans to associate the macro with input functions of the accessories(e.g., one or more keys of the keyboard 108, and one or more GUIelements 119 of the touch screen display).

In step 822, the AMS application can respond to drag and dropassociations of actions and input functions of the keyboard 108 andselectable GUI elements of the touch screen display 115. If anassociation is detected, the AMS application can proceed to step 824where it can determine if a profile has been identified in step 812 torecord the association(s) detected. If a profile has been identified,the associations are recorded/stored in the profile in step 826. If aprofile has not been identified in step 812, the AMS application cancreate a profile in step 828 for recording the detected associations. Inthe same step, the user can name the newly created profile as desired.The newly created profile can also be associated with one or more gamingsoftware applications in step 830 for future reference. It is noted thatin the case of the touch screen display 115, the AMS application recordsin step 826, each screen shot with the associated actions.

The AMS application can also store in the profile a gaming venue statefor each screen shot. The gaming venue state can be a description of agaming state (e.g., gaming controls for a tank), images of what thegaming venue state looks like (e.g., one or more still images of a tank,or a video of an avatar entering a tank), and/or application programminginstructions (API) messages which can be received from the gamingapplication to enable the AMS application to identify the occurrence ofa gaming venue state. The AMS application can be adapted to utilizeimage processing technology to detect a gaming venue state according topre-stored images or video clips in the profile. For example, the AMSapplication can use image processing technology to identify the avatarof the gamer and track what the avatar does as directed by the gamer.For example, if the avatar enters a tank, the image processingtechnology of the AMS application can detect a gaming venue state inwhich a screen shot of tank controls would be appropriate forpresentation at the touch screen display 115.

Referring back to step 826, once the associations have been recorded ina profile, the AMS application can determine in step 832 whether theaccessories (and/or the dongle 202) shown illustratively in FIGS. 2-4are available for programming. If the AMS application detects that theaccessories and/or the dongle 202 are communicatively coupled to thecomputer 204, the AMS application can proceed to step 834 of FIG. 8where it submits the profile and its contents for storage in one of theaccessories (e.g., the touch screen display 115 shown as tablet 302 inFIGS. 3-4) or the dongle 202 for recording. Alternatively, the AMSapplication can store the profile in the computing device (which may bean integral part of an accessory or the dongle 202).

The GUI 101 of FIG. 1 presented by the AMS application can have otherfunctions. For example, the GUI 101 can provide options for layout ofthe accessory selected (button 122), how the keyboard is illuminatedwhen associations between input functions and actions are made (button134), and configuration options for the accessory (button 126). The AMSapplication can adapt the GUI 101 to present more than one functionalGUI page. For instance, by selecting button 102, the AMS application canadapt the GUI 101 to present a means to create macros and associateactions to accessory input functions as depicted in FIG. 1. Selectingbutton 104 can cause the AMS application to adapt the GUI 101 to presentstatistics from raw stimulation information captured by the AMSapplication as will be described below. Selecting button 106 can alsocause the AMS application to adapt the GUI 101 to present promotionaloffers and software updates.

It should be noted that the steps of method 800 in whole or in part canbe repeated until a desirable pattern of associations of actions toinput functions of the selected accessories has been accomplished. Itwould be apparent to an artisan with ordinary skill in the art thatthere can be numerous other approaches to accomplish similar results.These undisclosed approaches are contemplated by the present disclosure.

FIG. 9 depicts a method 900 in which the AMS application can beprogrammed to recognize unknown accessories so that method 800 can beapplied to new accessories. Method 900 can begin with step 902 in whichthe AMS application detects an unknown accessory such as a new keyboard,tablet, or other type of accessory from an unknown vendor by way of acommunicative coupling to a computing device from which the AMSapplication operates. Assuming in this illustration the unknownaccessory is a keyboard, the AMS application in this instance canreceive identity information from the keyboard. The identity can be usedto search for the keyboard from a local or remote database. Upondetecting an unknown accessory, the AMS application in step 904 canpresent a depiction of an accessory of similar or same category inresponse to a user providing direction as to the type of accessory (byselecting for example a drop-down menu). Alternatively, or incombination with the user instructions, the AMS application candetermine from the identity information received from the unknownaccessory an accessory type.

In step 906 the AMS application can receive instructions describing allor a portion of the input functions of the unknown accessory. Theseinstructions can come from a user who defines each input functionindividually or responds to inquiries provided by the AMS application,or from a remote database that describes the features of the accessory.The AMS application can for example make an assumption as to thekeyboard layout and highlight each key with a proposed function whichthe user can verify or modify. Once the AMS application has beenprovided instructions in step 906, the AMS application can create anaccessory identity in step 908 which can be defined by the user. Insteps 910 and 912, the AMS application can associate and record theaccessory instructions with the identity for future recognition of theaccessory. In step 914, the AMS application can present a depiction ofthe new accessory with its identity along with the other selectableaccessories in section 117.

Method 900 can provide a means for universal detection andidentification of any accessory which can be used to control or managesoftware applications operating in a computing device.

FIG. 10 depicts a method 1000 for illustrating the operations of the AMSapplication for either of the configuration shown in FIGS. 3-4. In theconfigurations of FIGS. 3-4, the AMS application can be operating inwhole or in part from the tablet 302, the dongle 202, the gaming console306, or combinations thereof. For illustration purposes, it is assumedthe AMS application operates from the gaming console 306. Method 1000can begin with the AMS application establishing communications in steps1002 and 1004 with the gaming console 306 of FIGS. 3-4 and a gamingaccessory such as the tablet 302. These steps can represent for examplea user starting the AMS application in the gaming console 306 and/or theuser inserting at a USB port of the gaming console 302 a connector of aUSB cable tethered to the tablet 302. In step 1006, the tablet 302 canin turn provide the AMS application an accessory ID. With the accessoryID, the AMS application can identify in step 1008 a user accountassociated with the tablet 302. In step 1010, the AMS application canretrieve one or more profiles associated with the user account.

In step 1012, the user can be presented by way of a display coupled tothe gaming console 302 the profiles available to the user to choosefrom. If the user makes a selection, the AMS application proceeds tostep 1014 where it retrieves from the selected profiles theassociation(s) stored therein. If a selection is not made, the AMSapplication can proceed to step 1016 where it can determine whether asoftware gaming application (e.g., video game) is operating from thegaming console 302 or whether the gaming console 302 is communicatingwith the software gaming application by way of a remote systemcommunicatively coupled to the gaming console 302 (e.g., on-line gamingservers presenting World of Warcraft™). If a gaming software applicationis detected, the AMS application proceeds to step 1017 where itretrieves a profile that matches the gaming application detected and theassociation(s) contained in the profile. In the present context,association(s) can represent accessory stimulations, navigation, speech,the invocation of other software applications, or macros thereof. Theaccessory stimulations can be stimulations that are generated by thetablet 302 being used, stimulations from other unrelated accessories(e.g., a keyboard), or combinations thereof.

Once a profile and its contents have been retrieved in either of steps1014 or step 1017, the AMS application can proceed to step 1019 where itmonitors for a change in a gaming venue state based on the presentationsmade by the gaming application. At the start of a game, for example, thegaming venue state can be determined immediately depending on the gamingoptions chosen by the gamer. The AMS application can determine thegaming venue state by tracking the gaming options chosen, receiving anAPI instruction from the gaming application, or by performing imageprocessing on the video presentation generated by the gamingapplication. For example, the AMS application can detect that the gamerhas directed an avatar to enter a tank. The AMS application can retrievein step 1019 a screen shot associated with this gaming venue state asshown in FIG. 11, while the gaming application adjusts the presentationof the game at the monitor 1102 (e.g., view finder 1108 with a controlpanel 1110). In another illustration, the AMS application can detect anew gaming venue state as a result of that the gamer directing theavatar to leave the tank and travel on foot. Once again the AMSapplication retrieves in step 1019 a screen shot associated with thedetected gaming venue state and presents it at the tablet 302 as shownin FIG. 12. Contemporaneously, the gaming console 306 adjusts thepresentation of the game at the monitor 1202 in accordance with thegaming application.

Thus once the gaming venue state is detected in step 1018, the AMSapplication retrieves in step 1019 the screen shot associated with thedetected gaming venue state and presents the screen shot at the tablet302. The AMS application can then proceed to monitor in step 1020stimulations generated by the tablet 302. The stimulations can begenerated by the gamer by tactile touch of the touch screen GUI elementcontrols of the tablet 302. The AMS application can be adapted to detecta selection of a GUI element presented by the tablet 302 in severalways. For example, the tablet 302 can provide the AMS applicationinformation associated with the surface area covered by a gamer's fingerusing capacitive or resistive sensing technology. The greater thesurface area the more the gamer is depressing his/her finger on thetouch screen display of the tablet 302. Similarly, the less surface areadetected the less the gamer is depressing his/her finger on the tablet302. Detecting a large surface area or a sudden change in surface area(less to more can be an indicator to the AMS application that the gamerhas selected a GUI element. A continuous small surface area can beignored as a selection thus allowing the gamer to place several fingerson the selectable GUI elements of the tablet 302 without invoking aselection.

To assist the gamer, the AMS application can also be adapted to reservea portion of a presentation area of a monitor coupled to the gamingconsole 306 from which the gaming application is presented. In thereserved portion, the AMS application can replicate the screen shotpresented by the tablet 302 with an indication of which selectable GUIelements have a finger placed thereon by the gamer. This indication canbe shown by coloring the selectable GUI elements that have a fingerplace on it. FIG. 11 shows a tablet 302 tethered to a gaming console306, which in turn is tethered to a monitor 1102 from which the gamingapplication presents the game. In the bottom right corner, a copy of thescreen shot 1104 of the tablet 302 is presented with three buttonsshaded in gray 1106. The buttons shaded in gray represents buttons thatthe gamer has placed a finger. This visual aid provides a gamer a meansto visualize the game on the monitor 1102 and know the placement of thegamer's fingers on the tablet 302 without the gamer directing his/hereyes to the tablet 302. So long as the gamer lightly places his/herfingers on selectable GUI elements (buttons in the presentillustration), the AMS application will not detect that a selection hasbeen made. A similar presentation is shown in FIG. 12. In thisillustration the screen shot 1204 of the tablet 302 is at the upperright hand corner with the buttons shaded in gray 1206 based on thedetection of finger placement on each button shaded in gray.

In addition to detecting the surface area of a finger placed on thetablet 302, the AMS application can be adapted to detect where thesurface area is relative to the selectable GUI elements. If for example,the surface area of the finger is in a location on the tablet 302between selectable GUI elements, the AMS application can performpredictive and correlation analysis to determine which GUI element ismost likely of interest to the gamer. The AMS application can make thisdetermination from a proximity of the detected surface area of thefinger placement to one of the selectable GUI elements as well as thecontext of the game at the time the finger placement is detected. Forexample, the AMS application can determine that the finger placement isnear the fire button and farther from a communication button. The AMSapplication can also ascertain from the gaming application (via APImessages or image processing) that the avatar of the gamer is in themidst of receiving fire from enemy players. If the gamer depresseshis/her finger during this time such that a change in surface area isdetected (low surface area to high surface area), the AMS applicationcan predict that there's a higher likelihood the gamer is intending todepress the attack button rather than the communication button.

With the enhanced detection schemes described above, if a simulation isdetected at step 1020, the AMS application can determine in step 1022whether to pass the detected stimulation(s) to an Operating System (OS)of the gaming console 306 without alteration in step 1032 or withsubstitutions in steps 1026-1030. This determination can be made bycomparing the detected stimulation(s) to triggers associated with one ormore actions in the profile. If the detected stimulation(s) match thestimulations identified as triggers, then the AMS application proceedsto step 1026 where it retrieves substitute stimulation(s) from thetriggered action(s) in the profile. In step 1028, the AMS applicationsubstitutes the detected stimulation(s) with the substitute stimulationsand reports them to the OS of the gaming console 306. If there are nomatches detected in step 1022, the AMS application transmits in step1032 the detected stimulation to the OS of the gaming console 306without alteration. In step 1034, the OS determines whether to pass thereported stimulations to the gaming software application in step 1036,invoke in step 1038 a software application identified in thestimulation(s) reported in steps 1030 or 1032, or a combination thereof.

Contemporaneous to the embodiments described above, the AMS applicationcan record in step 1040 statistics or raw data relating to the detectedaccessory stimulations. A portion of the AMS application can operate asa background process which records statistics and/or raw data relatingto the stimulations detected. After completing a game, the user canselect button 104 in FIG. 1, which causes the AMS application to presentan updated GUI such as shown in FIG. 6, which illustrates the usage ofinput functions of one of the screen shots of the tablet 302 for whichstimulations were detected in step 1020 of FIG. 10.

In the illustration of FIG. 6, certain selectable GUI buttons arecolor-coded to illustrate the frequency of usage of these buttons. Acolor scale 603 defines the frequency of usage of the input functions ofthe tablet 302. The first end of the scale (navy blue) represents asingle detected depression, while an opposite end of the scale (brightred) represents 500 detected depressions. Based on this scale, the AMSapplication maps by color stimulations of the selectable GUI elements.For example, the grenade button has the highest detectable usage, whilethe engine has the fewest depressions. Buttons such as “Sensor” havingzero depressions are not color coded to readily identify the colormapping of keys which were used at least once.

The AMS application provides additional functions in a playback panel ofthe GUI which can help a gamer understand how the color coded keys wereused during an active software application such as a video game. In thissection of the GUI, the AMS application can present the user with aplayback control function 602 which the user can select to replay,pause, forward or rewind the usage of these keys. When usage playback isselected, the user can for instance see the color coded keys highlightedin real-time with a temporary white border to visualize how the keyswere selected. A time clock 604 provides the user the elapsed time ofthe playback sequence. Button 612 allows the user to retrieve statisticsfrom other sessions, while button 614 provides the user a means to savestatistics from a given session.

The GUI of FIG. 6 could have been shown as a split screen with allaccessories which generated one or more detected stimulations (e.g.,keyboard, mouse, and microphone), each providing statistical symbolicresults as described above for the keyboard. Although not shown, splitscreen embodiments are contemplated by the present disclosure for theGUI of FIG. 6.

In addition to a symbolic representation as shown in FIG. 6, the AMSapplication can provide the gamer a means to visualize raw statistics ina table format such as shown in FIG. 7 by selecting button 612. Thetable format shows raw data in section 702 and possible suggestions insection 704 for improving user performance which can be generated by theAMS application. Section 702 can be presented in a table format with acolumn identifying the key being analyzed, its usage, and number of keypresses. The user can ascertain from this table the most and leastfrequently used keys as well as other identifiable patterns.

The AMS application can utilize an understanding of the layout of theaccessory to determine from the statistics ways that the user canimprove response time or ergonomic use. For example, the AMS applicationcan determine from a layout analysis that the selectable GUI elementsrearranging the Grenade, GPS and Arm buttons may save the gamer time inexecution thereby speed the gamers reactions and improve performance.

The AMS application can utilize present and next generation algorithmsto determine how to improve response times and ergonomic usage ofaccessory devices. The AMS application can for example have at itsdisposal an understanding of the layout of each accessory, the type ofsoftware being controlled by the accessory, type of operations commonlyused to control the software (e.g., known actions as shown in theactions column 130 of FIG. 1), an understanding of the associations madeby other users (e.g., gamers) to improve their performance whencontrolling the software, and so on. The AMS application can also beadapted to communicate with the gaming application via an API to receiveadditional gaming statistics. The AMS application can also utilizestatistical and behavior modeling techniques to predict the behavior ofthe gamer and responses from the software application to identifypossible ways to improve the gamer's performance.

From the foregoing descriptions, it would be evident to an artisan withordinary skill in the art that the aforementioned embodiments can bemodified, reduced, or enhanced without departing from the scope andspirit of the claims described below. For example, the AMS applicationcan be integrated in the tablet 302 thereby performing all or a portionof methods 800-1000.

In another embodiment, the AMS application can be adapted to define morethan one programmable layer for an accessory. Such a feature can extendthe functionality of an accessory into multi-layer paradigms of inputfunctions. The GUI of FIG. 1 can be adapted so that a user can specifymore than one programmable layer for a specific accessory (e.g., thetouch screen display 115 or tablet 302. The user can also specify whichlayer to present in FIG. 1 while associating actions. If for instancelayer 1 is shown, the GUI of FIG. 1 can present the actions associatedin this layer by presenting descriptors superimposed on the inputfunctions (e.g., buttons or keys). When the user switches to layer 2(e.g., by selecting from a drop-down menu the layer of interest) theaccessory can be shown in the GUI with a different set of associatedactions. The user can define a macro or identify a key sequence toswitch between layers when the accessory is in use.

The trigger for switching between layers can be a toggle function (e.g.,selecting a layer button on the tablet 302) to switch between layers ina round robin fashion (layer 1→layer 2→layer 3→to layer 1→ and so on).Alternatively, the user can define a hold and release trigger to switchbetween layers. In this embodiment, the user moves to another layerwhile pressing a button on the tablet 302 and returns to the precedinglayer upon its release. In yet another embodiment, the trigger to switchlayers can be defined differently per layer. The user can for exampleselect one button in layer 1 to proceed to layer 2, and select adifferent button in layer 2 to return to layer 1 or proceed to yetanother layer 3. There can be numerous combinations of layers andtriggers which can be defined to substantially expand the capability ofsingle accessory.

In another embodiment, the AMS application can be adapted so that agamer can define super macros and/or super profiles. A super macro canrepresent nested macros (combinations of macros). AMS application can beadapted so that the gamer can customize the timing for executing nestedmacros. Similarly, a super profile can represent nested profiles(combinations of profiles). A super profile can for example comprisesub-profiles, each sub-profile defining associations of actions to inputfunctions of a particular accessory. The super macros and/or profilescan be programmed into the dongle 202.

In yet another embodiment, the tablet 302 can be equipped with abiometric device such as a fingerprint reader to identify the gamer andthereby provide an identity that describes the gamer and the tablet 302used thereby. In this embodiment, different users can utilize the sametablet 302 but invoke a different user account depending on thebiometric data provided with the accessory ID.

The foregoing embodiments are a subset of possible embodimentscontemplated by the present disclosure. Other suitable modifications canbe applied to the present disclosure.

FIG. 13 depicts an exemplary diagrammatic representation of a machine inthe form of a computer system 1300 within which a set of instructions,when executed, may cause the machine to perform any one or more of themethods discussed above. One or more instances of the machine canoperate, for example, as the tablet 302, the gaming console 306 andother devices of FIGS. 2-4.] In some embodiments, the machine may beconnected (e.g., using a network) to other machines. In a networkeddeployment, the machine may operate in the capacity of a server or aclient user machine in server-client user network environment, or as apeer machine in a peer-to-peer (or distributed) network environment.

The machine may comprise a server computer, a client user computer, apersonal computer (PC), a tablet PC, a smart phone, a laptop computer, adesktop computer, a control system, a network router, switch or bridge,or any machine capable of executing a set of instructions (sequential orotherwise) that specify actions to be taken by that machine. It will beunderstood that a communication device of the present disclosureincludes broadly any electronic device that provides voice, video ordata communication. Further, while a single machine is illustrated, theterm “machine” shall also be taken to include any collection of machinesthat individually or jointly execute a set (or multiple sets) ofinstructions to perform any one or more of the methods discussed herein.

The computer system 1300 may include a processor 1302 (e.g., a centralprocessing unit (CPU), a graphics processing unit (GPU, or both), a mainmemory 1304 and a static memory 1306, which communicate with each othervia a bus 1308. The computer system 1300 may further include a videodisplay unit 1310 (e.g., a liquid crystal display (LCD), a flat panel,or a solid state display. The computer system 1300 may include an inputdevice 1312 (e.g., a keyboard), a cursor control device 1314 (e.g., amouse), a disk drive unit 1316, a signal generation device 1318 (e.g., aspeaker or remote control) and a network interface device 1320.

The disk drive unit 1316 may include a tangible computer-readablestorage medium 1322 on which is stored one or more sets of instructions(e.g., software 1324) embodying any one or more of the methods orfunctions described herein, including those methods illustrated above.The instructions 1324 may also reside, completely or at least partially,within the main memory 1304, the static memory 1306, and/or within theprocessor 1302 during execution thereof by the computer system 1300. Themain memory 1304 and the processor 1302 also may constitute tangiblecomputer-readable storage media.

Dedicated hardware implementations including, but not limited to,application specific integrated circuits, programmable logic arrays andother hardware devices can likewise be constructed to implement themethods described herein. Applications that may include the apparatusand systems of various embodiments broadly include a variety ofelectronic and computer systems. Some embodiments implement functions intwo or more specific interconnected hardware modules or devices withrelated control and data signals communicated between and through themodules, or as portions of an application-specific integrated circuit.Thus, the example system is applicable to software, firmware, andhardware implementations.

In accordance with various embodiments of the present disclosure, themethods described herein are intended for operation as software programsrunning on a computer processor. Furthermore, software implementationscan include, but not limited to, distributed processing orcomponent/object distributed processing, parallel processing, or virtualmachine processing can also be constructed to implement the methodsdescribed herein.

While the tangible computer-readable storage medium 622 is shown in anexample embodiment to be a single medium, the term “tangiblecomputer-readable storage medium” should be taken to include a singlemedium or multiple media (e.g., a centralized or distributed database,and/or associated caches and servers) that store the one or more sets ofinstructions. The term “tangible computer-readable storage medium” shallalso be taken to include any non-transitory medium that is capable ofstoring or encoding a set of instructions for execution by the machineand that cause the machine to perform any one or more of the methods ofthe present disclosure.

The term “tangible computer-readable storage medium” shall accordinglybe taken to include, but not be limited to: solid-state memories such asa memory card or other package that houses one or more read-only(non-volatile) memories, random access memories, or other re-writable(volatile) memories, a magneto-optical or optical medium such as a diskor tape, or other tangible media which can be used to store information.Accordingly, the disclosure is considered to include any one or more ofa tangible computer-readable storage medium, as listed herein andincluding art-recognized equivalents and successor media, in which thesoftware implementations herein are stored.

Although the present specification describes components and functionsimplemented in the embodiments with reference to particular standardsand protocols, the disclosure is not limited to such standards andprotocols. Each of the standards for Internet and other packet switchednetwork transmission (e.g., TCP/IP, UDP/IP, HTML, HTTP) representexamples of the state of the art. Such standards are from time-to-timesuperseded by faster or more efficient equivalents having essentiallythe same functions. Wireless standards for device detection (e.g.,RFID), short-range communications (e.g., Bluetooth, WiFi, Zigbee), andlong-range communications (e.g., WiMAX, GSM, CDMA, LTE) are contemplatedfor use by computer system 1300.

The illustrations of embodiments described herein are intended toprovide a general understanding of the structure of various embodiments,and they are not intended to serve as a complete description of all theelements and features of apparatus and systems that might make use ofthe structures described herein. Many other embodiments will be apparentto those of skill in the art upon reviewing the above description. Otherembodiments may be utilized and derived therefrom, such that structuraland logical substitutions and changes may be made without departing fromthe scope of this disclosure. Figures are also merely representationaland may not be drawn to scale. Certain proportions thereof may beexaggerated, while others may be minimized. Accordingly, thespecification and drawings are to be regarded in an illustrative ratherthan a restrictive sense.

Although specific embodiments have been illustrated and describedherein, it should be appreciated that any arrangement calculated toachieve the same purpose may be substituted for the specific embodimentsshown. This disclosure is intended to cover any and all adaptations orvariations of various embodiments. Combinations of the aboveembodiments, and other embodiments not specifically described herein,are contemplated by the present disclosure.

The Abstract of the Disclosure is provided with the understanding thatit will not be used to interpret or limit the scope or meaning of theclaims. In addition, in the foregoing Detailed Description, it can beseen that various features are grouped together in a single embodimentfor the purpose of streamlining the disclosure. This method ofdisclosure is not to be interpreted as reflecting an intention that theclaimed embodiments require more features than are expressly recited ineach claim. Rather, as the following claims reflect, inventive subjectmatter lies in less than all features of a single disclosed embodiment.Thus the following claims are hereby incorporated into the DetailedDescription, with each claim standing on its own as a separately claimedsubject matter.

What is claimed is:
 1. An apparatus, comprising: a memory to storeexecutable instructions; and a processing system including a processor,the processing system being coupled to the memory, wherein theprocessing system, responsive to executing the instructions, performsoperations comprising: determining, according to a profile, a pluralityof game-related screen shots, each portraying a plurality of graphicuser interface elements for presentation at a display, wherein theplurality of graphic user interface elements is associated with aplurality of gaming application actions according to a firstassociation; and determining a second association between each of theplurality of game-related screen shots and a corresponding gaming venuestate of a plurality of gaming venue states.
 2. The apparatus of claim1, wherein the operations comprise determining that a video gamingsystem presenting a video game has presented a first gaming venue stateof the plurality of gaming venue states.
 3. The apparatus of claim 1,wherein the operations comprise: selecting a first game-related screenshot of the plurality of game-related screen shots corresponding to afirst gaming venue state according to the second association; andpresenting the first game-related screen shot at the display, whereinthe first game-related screen shot portrays a first plurality of graphicuser interface elements.
 4. The apparatus of claim 1, wherein theoperations comprise: detecting a stimulation of a selected one of afirst plurality of graphic user interface elements; and transmitting afirst one of the plurality of gaming application actions correspondingto the selected one of the first plurality of graphic user interfaceelements according to the first association.
 5. The apparatus of claim1, wherein the operations comprise determining that a video gamingsystem has transitioned to a presentation of a second gaming venue stateof the plurality of gaming venue states.
 6. The apparatus of claim 1,wherein the operations comprise: identifying a second game-relatedscreen shot of the plurality of game-related screen shots correspondingto a second gaming venue state according to the second association; andpresenting the second game-related screen shot at the display, whereinthe second game-related screen shot portrays a second plurality ofgraphic user interface elements.
 7. The apparatus of claim 1, whereinthe operations comprise: detecting a second stimulation of a selectedone of a second plurality of graphic user interface elements; andtransmitting a second one of the plurality of gaming application actionscorresponding to the selected one of the second plurality of graphicuser interface elements according to the first association.
 8. Amachine-readable storage device, comprising executable instructions,which responsive to being executed by a processing system including aprocessor, cause the processing system to perform operations comprising:determining a plurality of game-related screen shots, each comprising aplurality of graphic user interface elements for presentation at adisplay, wherein a plurality of gaming application actions is associatedwith the plurality of graphic user interface elements according to afirst association; determining a second association between each of theplurality of game-related screen shots and a corresponding gaming venuestate of a plurality of gaming venue states; initiating communicationswith a video gaming system; and determining that the video gaming systemhas presented a first gaming venue state of the plurality of gamingvenue states, wherein the determining that the video gaming system haspresented the first gaming venue state is based upon an image, stored ina profile, corresponding to the first gaming venue state.
 9. Themachine-readable storage device of claim 8, wherein the operationscomprise: identifying a first game-related screen shot of the pluralityof game-related screen shots corresponding to the first gaming venuestate of the plurality of gaming venue states according to the secondassociation; and presenting at the display the first game-related screenshot comprising a first plurality of graphic user interface elements.10. The machine-readable storage device of claim 8, wherein theoperations comprise: receiving a first stimulation responsive to aselection of a first selected one of a first plurality of graphic userinterface elements; determining a first selected one of the plurality ofgaming application actions corresponding to the first selected one ofthe first plurality of graphic user interface elements according to thefirst association; and transmitting the first selected one of theplurality of gaming application actions to the video gaming system tocontrol a presentation of a video game.
 11. The machine-readable storagedevice of claim 8, wherein the operations comprise detecting that thevideo gaming system has transitioned to a presentation of a secondgaming venue state of the plurality of gaming venue states.
 12. Themachine-readable storage device of claim 8, wherein the operationscomprise: identifying a second game-related screen shot of the pluralityof game-related screen shots corresponding to a second gaming venuestate according to the second association; and presenting, at thedisplay, the second game-related screen shot portraying a secondplurality of graphic user interface elements.
 13. The machine-readablestorage device of claim 8, wherein the operations further comprise:receiving a second stimulation responsive to a selection of a secondselected one of a first plurality of graphic user interface elements;determining a second selected one of the plurality of gaming applicationactions corresponding to the second selected one of the first pluralityof graphic user interface elements according to the first association;and transmitting the second selected one of the plurality of gamingapplication actions to the video gaming system to further control thepresentation of a video game.
 14. A method, comprising: determining, bya processing system including a processor, a plurality of game-relatedscreen shots, each comprising a plurality of graphic user interfaceelements, wherein a plurality of gaming application actions isassociated with the plurality of graphic user interface elementsaccording to a first association; determining, by the processing system,a second association between each of the plurality of game-relatedscreen shots and a corresponding gaming venue state of a plurality ofgaming venue states; and detecting, by the processing system, that agaming system has presented a first gaming venue state of the pluralityof gaming venue states, wherein the detecting that the gaming system haspresented the first gaming venue state comprises image processing of avideo game presentation of the gaming system.
 15. The method of claim14, comprising: identifying, by the processing system, a firstgame-related screen shot of a first plurality of game-related screenshots according to the first gaming venue state and the secondassociation; and presenting, by the processing system, the firstgame-related screen shot, wherein the first game-related screen shotcomprises a first plurality of graphic user interface elements of theplurality of graphic user interface elements.
 16. The method of claim14, comprising: detecting, by the processing system, a selection of afirst graphical user interface element selected from the plurality ofgraphic user interface elements; selecting, by the processing system, afirst gaming application action of the plurality of gaming applicationactions according to the selection of the first graphical user interfaceelement; and transmitting the first gaming application action to thegaming system to control the video game presentation.
 17. The method ofclaim 14, further comprising: detecting, by the processing system, aselection of a second graphical user interface element selected from theplurality of graphic user interface elements; selecting, by theprocessing system, a second gaming application action of the pluralityof gaming application actions according to the second graphic userinterface element; and transmitting the second gaming application actionto the gaming system to further control the video game presentation. 18.The method of claim 14, further comprising: detecting, by the processingsystem that the gaming system has transitioned to a presentation of asecond gaming venue state of the plurality of gaming venue states;identifying, by the processing system, a second game-related screen shotof the plurality of game-related screen shots corresponding to thesecond gaming venue state according to the second association; andpresenting, by the processing system, the second game-related screenshot comprising a second plurality of graphic user interface elements.19. The method of claim 14, further comprising: receiving, by theprocessing system, an association of an associable gaming applicationaction of a plurality of associable gaming application actions with oneof the plurality of graphic user interface elements portrayed on one ofthe plurality of game-related screen shots.
 20. The method of claim 14,further comprising: detecting a surface area of a finger; and detectinga selection of one of the two of a first plurality of graphic userinterface elements according to: an image presentation of the video gamepresentation during the first gaming venue state; and the surface areaof the finger.